ASC  Engineering  Directorate 


Integrity  -  Service  -  Excellence 


Revitalization  of  Manufacturing 
and  Quality  Assurance  within 

Systems  Engineering 


Dr.  Thomas  F.  Christian, 
Richard  Stepler,  Hamid  Akhbari 
Aeronautical  Systems  Center 
SSTC,  21  June  2007 


Manufacturing  &  Quality  in  Systems  Engineering 


•  Why  are  we  here? 

•  Are  there  really  deficiencies  in  our  Systems  Engineering  Process  or  is  there  a 
problem  in  execution  ? 

•  What  metrics  are  our  critics  using  to  gauge  our  performance? 

•  Failures  to  make  rate,  cost,  and  schedule  in  production 

•  Unresolved  engineering  issues  from  the  development  phase  manifest 
themselves  as  cost  and  rate  failures  detected  during  LRIP  and  Production 

•  Fixing  the  root  cause  of  manufacturing  problems  means  including 
manufacturing  as  a  mandatory  discipline  in  the  System  Engineer  tool  set 
during  design  and  development 

•  My  perspective  on  the  Past,  Present,  and  Future  of  Manufacturing 
and  Quality  involvement  in  Systems  Engineering 


Overview 


•  Where  we  were  ► 


•  Where  we  are  ► 


•  Where  we  are  going  ► 


Where  We  Were:  M&Q  in  America 


Ahhh. .  ..’’The  Good  Old  Days”? 

. Were  they  as  “good”  as  we  remember. . . 

Separation  of  design  and  manufacturing  functions 

•  Transition  to  production  always  problematic 

•  LRIP  created  to  address  problem  but  only  addressed  symptoms 

•  Major  redesigns  of  components  required  to  achieve  desired  production 
rates 

•  Producibility  changes  euphemism  for  “we  can’t  afford  to  build  what 
you  designed” 

•  Cost  high:  low  first  pass  yields  and  traveled  work 

•  Schedule  fluctuations  due  to  excessive  “work  in  process” 

Quality  by  inspection 

•  The  most  expensive  and  least  effective  approach 

•  Build-test- fix-retest-fix-retest - who  pays  for  quality 


Where  We  Were:  M&Q  in  America 


•  Need  proof? 

•  Back  in  the  1970s,  how  long  did  your  domestic  car  last? 

•  Corporate  commitment  to  quality  and  the  customer’s  satisfaction? 

•  “What’s  good  for  General  Motors  is  good  for  the  country” 

•  Then  came  competition  from  Japan,  with  help  from  Deming 

•  The  American  auto  industry  wakes  up 

•  Recognized  Japan’s  focus  on  customer  satisfaction  and  quality 

•  Today:  we  expect  our  cars  to  work  every  time. .  ..all  the  time 

•  Toyota  is  still  at  the  forefront  of  quality 

•  Their  “secret”  -  focus  on  quality  and  producibility  during  design 


THE  W.  EDWARDS  DEMING  INSTITUTE® 


Where  We  Are:  M&Q  in  America 


FACT:  The  American  Defense  Aerospace  Industry  has  produced, 
unquestionably,  the  finest  weapon  systems  in  the  world 

•  But  at  what  cost?  How  long  does  it  take?  How  much  “cost  of 
quality”  have  we  shifted  to  operations  and  maintenance? 


“...DoD  is  simply  not  positioned  to  deliver  high-quality  products  in  a  timely  and 
cost-efficient  fashion.” 

GAO  (HASC)  Testimony  ‘Actions  Needed  to  Get  Better  Results  on  Weapons  Systems 

Investments”  5  April  2006 


“In  2001,  The  Average  Weapon  System  Acquisition  Program  Experienced  a 
36%  Cost  Overrun  and  Schedule  Delay  of  Two  Years”  -  Dr.  Marvin  Sambur 


M&Q  in  the  DoD: 

Who  We  Were  Then . and  Who  We  Are  Now 


•  THEN. ...1975  to  1985 

•  M&Q  Organization 

•  Every  level  of  acquisition  decision  authority  in  DoD 

•  Independent  M&Q  evaluation  with  veto  power 

•  Large  numbers  of  experienced  people 

•  Career  field  with  opportunities 

•  Mentoring 

•  Tribal  knowledge  well  documented  in  Specs  and  Standards 

•  NOW . 2001  to  present 

•  Not  sitting  at  the  decision  table 

•  No  seat  and  no  vote  on  program  readiness 

•  Under-represented  at  most  locations 

•  No  representation  at  some  locations 

•  M&Q  Specs  and  Standards  cancelled 


GAO  Findings: 
Production  Maturity 


□  Program  is  not  taking  | - ,  Taking  steps  (0  mea 

steps  to  meet  GAO  production  maturity  criteria 

production  maturity  criteria 


Program  demonstrates 
sufficient  production  maturity 


Program 
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2003 
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Color  ratings  based  on  GAO  opinions 
Source:  GAO  Quick  Look  reports  for 


n/a  =  not  assessed 


Defense  Science  Board  ManTech  Study 


DSB  was  tasked  by  SAF/AQ  to  evaluate  the  ManTech  program 

Released  report  in  February  06 

Much  of  the  report  pertains  specifically  to  ManTech 

Portion  of  the  report  addresses  global  acquisition 
manufacturing  issues 

•  Assessing  program  readiness  for  production . (suggested 

using  the  new  Manufacturing  Readiness  Levels  (MRLs), 
more  on  the  MRLs  Later) 

•  Workforce  Expertise  -  clearly  addresses  the  entire  DoD 
acquisition  workforce 

Complete  DSB  report  is  located  at: 


Defense  Science  Board  ManTech  Study 


Findings: 

•  Manufacturing  talent  in  the  DoD  workforce,  and  its  supporting  industrial 
base,  has  and  continues  to  decline 

•Not  enough  people  (both  at  working  level  and  in  leadership  positions) 
understand  the  processes  involved  in  developing  and  manufacturing  defense 
systems 

Recommendations  to  correct  knowledge  deficiency: 

•  Create  policy  requiring  support  for  programs  such  as  ManTech 

•  Implementation  of  MRLs  as  part  of  DoDI  5000.2 


Customer  Expectations 


1 .  Avoid  cost  overruns,  performance  shortfalls,  and 
schedule  slips  typically  manifested  during  production 

2.  Improve  quality  and  avoid  surprises 

3.  Ensure  affordability  and  producibility 

4.  Identify  all  potential  MFG  risks  during  transition  from 
development  to  production  and  establish  risks 
mitigation  plans 

5.  Provide  rapid  response  to  emerging  needs,  e.g., 
readiness  (includes  combat  ops,  surge,  parts  and 
spares,  etc.) 


Have  you  met  your  customers  expectations? 


Swinging  Pendulum  of  Acquisition  Reform 


•Where  we  were  was  not  cheap. . . .but  it  was  defined 

•Where  we  are  is  neither  cheap  nor  defined 

•The  faster.. better.. cheaper  “acquisition  reform”  pendulum...  for 
us  a  wrecking  ball. .  ..left  M&Q  vulnerable 

•We  can  anticipate  the  return  swing  but  must  not  let  it  drive  us  back 
to  the  old  approach 


1996 


2006 


•. .  .a  narrow  approach  to  QA,  based  on 
inspections,  reviews,  and  documentation 
audits 

•  Numerous  Mfg/QA  MIL-Specs  put  on 
contract  (MIL-Q-9858A,  MIL-STD- 
1528,  MIL-STD-1520,  MIL-STD-1535) 

•  Numerous  deliverable  data  items 
(Mfg  Plan,  QA  Plan,  Line  of  Balance, 
Quality  Status  Reports,  etc.) 

•  Army  of  DCMA  inspectors 


•  Minimal  Government  oversight  of 
contractor  QA  and  Manufacturing  processes 

•  Minimal  contractual  requirements 
•No  dedicated  representation  in 


Mfg/QA  at  OSD 


Let’s  tailor  the  “return  trip” 


ASC/ENSM  Response  to  Customer  Concerns 


•  ASC/ENSM  plans  to  conduct  a  360°  evaluation  to  identify  solutions 
and  best  practices 


Requirements  Customers 

(SAF/AQ)  (Users’  Experience) 


Industry  Benchmarking 
(Airlines,  Toyota,  etc.) 


Competitive  Assessment 
(Navy,  Army  Acquisition) 


Prime  Contractors 


Big  Industry  Messages 


1 .  Government  acquisition  strategies  do  not  require  an 
in-depth  risk  analysis  for  manufacturing  during  product 
design. 

2.  Government  does  not  specify  the  right  deliverables  in 
their  contracts. 

3.  Benchmark  other  industries  to  get  a  better  picture  on 
MFG  related  issues  during  product  development  and 
risk  mitigation  plans  to  address  them. 

4.  Assess  production  readiness  in  a  meaningful  way. 

5.  More  emphasis  on  suppliers  during  product 
development. 


Big  Industry  Message 


1 .  Government  acquisition  strategies  do  not  require  an 
in-depth  risk  analysis  for  manufacturing  during  product 
design: 

•  Establish  effective  source  selection  criteria  to  emphasize 
producibility  and  affordability 

•  Identify  incentives  for  contractors  to  focus  on  producibility  and 
affordability  during  product  development. 

•  More  MFG/QA  emphasis  during  ASP  reviews 

•  Strong  Government  advocate/champion  are  needed 

•  There  is  a  gross  lack  of  knowledge  and  personnel  in  this  area 

•  Hold  PMs  and  chief  engineers  accountable 

•  Educate  Government  PMs  with  potential  MFG/QA  risks  and 
their  impacts  to  the  overall  system  life  cycle  cost 

•  Make  long-term  decisions  thoroughly  considering  all 
production  risks  down  the  road 


Big  Industry  Message 


2.  Government  does  not  specify  the  right  deliverables  in 
their  contracts: 

•  Government  needs  to  verify  that  the  contractor  has  the  right 
processes  in  place  to  deliver  the  right  product 

•  Government  does  not  use  the  right  metrics  to  measure 
performance 

•  Make  the  contractor  demonstrate  that  they  have  a  solid 
production  plan 

•  Require  the  prime  to  demonstrate  control  of  MFG  processes 
during  development 

•  Specify  proper  MFG/QA  contractual  requirements  in 
development  contracts 


Big  Industry  Message 


3.  Benchmark  other  industries  to  get  a  better  picture  on 
MFG  related  issues  during  product  development  and 
risk  mitigation  plans  to  address  them: 

•  Consider  world-class  performers  in  other  industries 

•  Think  outside  the  box 

•  Develop  lessons  learned 

•  Evaluate  commercial  programs  and  practices  as  well  as  the 
FAA 

•  Consider  having  budget  for  “Producibility  Improvement  Plan” 
(PIP) 


Big  Industry  Message 


4.  Assess  production  readiness  in  a  meaningful  way: 

•  Government  needs  to  develop  better  MFG  transition 
strategies 

•  Willoughby  templates  (Transition  from  Development  to 
Production)  are  useful  tools 

•  PRRs  are  not  focusing  on  the  right  parameters.  Many 
programs  do  not  conduct  full  blown  PRRs  like  they  once  did 

•  MRLs  will  be  a  useful  tool  once  up  and  running 

•  Government  PMs  should  be  required  to  develop  MFG  exit 
criteria  for  milestone  reviews 

•  Industry  recognizes  “Production  Plans”  once  required  by  the 
Government  for  most  programs  as  a  useful  tool 


Big  Industry  Message 


More  emphasis  on  suppliers  during  product 
development: 

The  vast  majority  of  quality  related  issues  come  from  lower- 
tier  suppliers.  Ensure  that  the  prime’s  processes  for 
management  of  their  suppliers  are  solid 

Properly  manage  requirements  flow-down  to  lower-tier 
suppliers 

Require  suppliers  participation  on  IPTs  during  product 
development 

Ensure  supplier  participation  in  the  systems  engineering 
process,  in  particular  MFG  processes  and  procedures 

Develop  predictive  indicators  to  assess  supplier’s  “internal 
health” 

Use  of  common  metrics 


Where  We  Should  Be  Going 


•  The  way  forward . 

•  Internally:  Manufacturing  and  Quality  must  be  the  responsibility  of  design 
engineers  and  be  considered  early  in  the  development  process 

•  Externally:  Supplier  Management. . .  .engineers  at  primes  must  partner  with 
suppliers  to  achieve  maximum  affordability 

•  To  help  with  all  of  it:  The  M&Q  tool  set: 

•  Manufacturing  Development  Guide- Available  now 

•  Manufacturing  Readiness  Levels-Draft  available  now 

•  Manufacturing  and  Quality  Integrity  Program- Available  soon 


Systems  Engineering  -  Ensure  that  design  meets 

requirements  and  is  producible 

Mfg/QA  helps  SE  meet  producibility,  OSS&E,  and  Airworthiness 

Design  requirements 


Producibility. .  .M&Q’s 
Contribution  to  Systems  Engineering 


America’s  Defense  Aerospace  Industry  is  #1  in  the  world. 

However. . . 

•  Are  we  be  able  to  buy  desired  quantities? 

•  How  many  B-2s  were  originally  planned?  F-22s?  JSFs? 

If  we  can’t  figure  out  a  way  to  build  better  systems  cheaper,  we  will 
fulfill  Norm  Augustine’s  prophecy: 

•  "In  the  year  2054,  the  entire  defense  budget  will  purchase  just  one  aircraft.  This 
aircraft  will  have  to  be  shared  by  the  Air  Force  and  Navy  3-1/2  days  each  per 
week  except  for  leap  year,  when  it  will  be  made  available  to  the  Marines  for  the 
extra  day.  ” 

My  primary  focus  is  to  integrate  quality  and  producibility  early  in  the 
Systems  Engineering  process  (see  the  ASC,  MDG) 

More  information  on  the  Manufacturing  Development  Guide  (MDG) 
is  available  at:  http://engineering.wpafb.af.mil/mdg/mdg.asp 


Summary:  We  Know  We  Are  There  When 


•  The  process  used  to  manufacture  a  part  is  given  equal 
consideration  as  the  functionality  of  the  part 

•  Product  performance  and  producibility  are  equal  in  the  risk 
analysis  trade  studies  during  product  development 

•  The  “chiefs”  of  design,  manufacturing  and  logistics  have  equal 
votes  on  critical  design  decisions  during  development 

•  “Design”  executives  are  held  accountable  for  unit  production  cost 
and  cost  of  quality  decisions 

•  M&Q  metrics  are  present  in  entry  and  exit  criteria  for  each  phase 
of  the  acquisition  life  cycle 

•  Integrating  people,  processes,  and  technology  using  the  System 
Engineering  Process  proven  effective  by  world  class  producers 


As  good  systems  engineers,  our  commitment  to 

M&Q  starts  in  design 


Thank  you  for  your  time 
and  attention 

Thomas.Christian@wpafb.af.mil 

Phone:937-255-1826 


